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STATEMENT OF FOCUS 



The Wisconsin Research and Development Center for Cognitive Learning 
focuses on contributing to a better understanding of cognitive learning by 
children and youth and to the improvement of related educational practices. 
The strategy for research and development is comprehensive. It includes 
basic research to generate new knowledge about the conditions and processes 
of learning and about the processes of instruction, and the subsequent de- 
velopment of research-based instructional materials, many of which are de- 
signed for use by teachers and others for use by students. These materials 
are tested and refined in school settings. Throughout these operations be- 
havioral scientists, curriculum experts, academic scholars, and school people 
interact, insuring that the results of Center activities are based soundly 
on knowledge of subject matter and cognitive learning and that they are 
applied to the improvement of educational practice. 

This. Technical Report is from the Situational Variables and Efficiency 
of Concept Learning Project in Program 1. General objectives of the Pro- 
gram are to generate new knowledge about concept learning and cognitive 
skills, to synthesize existing knowledge, and to develop educational ma- 
terials suggested by the prior activities. Contributing to these Program 
objectives, the Concept Learning Project has the following five objectives: 
to identify the conditions that facilitate concept learning in the school 
setting and to describe their management, to develop and validate a schema 
for evaluating the student's level of concept understanding, to develop and 
validate a model of cognitive processes in concept learning, to generate 
knowledge concerning the semantic components of concept learning, and to 
identify conditions associated with motivation for school learning and to 
describe their management. 
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ABSTRACT 



The object of this study was to determine the effect of the number 
of instances and the emphasis of relevant attribute values on the level 
of concept mastery. 

Eight versions of programed lessons dealing with geometric concepts 
were prepared in which the independent variables of number of instances 
(4 or 8) and emphasis of relevant attribute values (presence or absence 
of attention-directing and review questions) were varied systematically. 
The subjects, who were fourth- and sixth-grade children, studied these 
lessons approximately 20 minutes a day for four days. 

After study of the lessons, the children were given a multiple- 
choice test and a completion test, each consisting of eleven types of 
questions related to concept learning. These questions were formulated 
to test recognition and production of attribute examples, attribute 
names, concept examples and non-examples, concept names, relevant and 
irrelevant attributes, concept definitions, and relationships of the 
concepts to one another. 

The essential findings of the study were: 

(1) Increasing the number of instances from 4 to 8 did not signifi- 
cantly affect overall concept mastery for either fourth- or sixth- 
grade children. 

(2) Increasing the number of instances significantly improved recogni- 
tion of concept non-examples for fourth-grade children. 
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(3) Emphasis of 
all concept 

overall concept mastery fot sixth-grade children was not significant 

(4) Emphasis of relevant attribute values significantly increased recog- 
nition and production of attribute names for fourth-grade children 
and recognition of attribute names for sixth-grade children. 



relevant attribute values significantly increased over- 
mastery for fourth-grade children. The increase in 
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Chapter I 
INTRODUCTION 



Concept learning is a topic of gre^.t theoretical interest and 
practical importance. Glaser (1968) has suggested that concepts are 
based on simple behavioral acts and form the elements of higher-order 
behaviors. This position of intermediate complexity provides an 
ideal situation to study the interplay of various elemental processes 
in the accomplishment of learning outcomes. Laboratory studies by 
many persons affiliated with the Wisconsin Research and Development 
Center for Cognitive Learning have contributed to a better understand 
ing of the stimulus variables and cognitive operations related to con 
cept learning (Fredrick, 1966, 1968; Jones, 1968; Klausmeier, Harris, 
Davis, Schwenn, & Frayer, 1968; Lemke, Klausmeier, & Harris, 1967; 
Lynch, 1966; Ramsay, 1965; Smuckler, 1966). Further, concepts com- 
prise a large and important segment of the subject matter taught in 
the schools. Guidelines for effective concept teaching can therefore 
make a significant contribution to improved learning in a wide range 
of situations. For this reason, personnel at the Center have also 
extended concept learning principles through research in school set- 
tings (Blount, Klausmeier, Johnson, Fredrick, & Ramsay, 1967; Kalish, 
1966; Kennedy, 1968; Steffe, 1966; Steffe & Parr, 1968). 
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A sizeable body of knowledge has accrued concerning laboratory 
concept learning. This research has made notable contributions to 
psychological theory and has indicated that certain instructional 
variables have powerful effects on concept attainment. Laboratory 
research, however, has been restricted in certain respects. This 
has limited its potential contribution to both theory and practice. 

Laboratory experiments have generally utilized specially-chosen 
concepts embodying only a few of the aspects which may influence learn- 
ing of concepts encountered in daily life. Most concepts studied in 
the laboratory are comprised of characteristics already known to the 
learner. Thus, the effect of variables influencing the learning of 
those characteristics is minimized. Laboratory concepts usually do 
not have meaningful concept labels, even though associations between 
such labels may have a powerful effect on transfer and interference 
in learning. Also, a large amount of response learning may be re- 
quired to acquire the labels. In order to examine aspects of con- 
cept learning such as these, a wider range of concepts should be 
employed. To permit meaningful comparisons, the concepts should be 
analyzed to determine their relevant characteristics and how these 
characteristics are combined, their irrelevant characteristics, the 
associative structure of the concept label, and the relationship of 
the concept to other concepts. In this manner, many kinds of concepts 
could be studied but their similarities and differences specified. 

A second way in which concept research has been restricted is in 
techniques of instruction. Typically, only concept examples and non- 
examples have been presented, the sequence of presentation being 
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determined by either the experimenter or the subject. Few studies 
have dealt with concept learning by definition, synonyms, or sentence 
context. Additional consideration should also be given to the effect 
of advance organizers, questions, and review. 

Measurement of concept learning has also focused on only a small 
segment of behaviors by which learning might be inferred. Trials, 
errors, or time required to reach a criterion of successive correct 
identifications have been commonly used measures. These techniques, 
however, do not assess the verbal aspects of concept learning such as 
labeling or definition. Nor do they test knowledge of relationships 
between concepts. Yet much of the power and utility of concept learn- 
ing derives from the possibility of communicating by naming or describ- 
ing and from relating the concept to other concepts. A set of behaviors 
which would permit inference of these various aspects of concept attain- 
ment would, therefore, be more informative than a single measure. It is 
probable that a particular instructional technique may facilitate some 
aspects of concept learning more than others. Also, past research has 
for the most part measured rate of acquisition rather than long-term 
retention. Both should be assessed. 

A final limitation of past concept learning research has been the 
preponderance of studies employing young adults as subjects. In order 
to discover possible developmental trends in concept learning, experi- 
ments should be replicated at various age levels. 

In summary, concept learning research should be extended in sev- 
eral ways: (1) a wider range of concepts should be examined, with 
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careful specification of the essential characteristics of the con- 
cepts; (2) various instructional procedures should be utilized, in- 
cluding verbal as well as nonverbal strategies; (3) a set of differen- 
tiated response measures should be employed to assess both short-term 
and long-term retention; and (4) performance of subjects of different 
ages on the same task should be compared. Simultaneous extension of 
these four aspects of concept learning research may reveal the cog- 
nitive processes entailed in concept learning and permit description 
of the interactive effect of concept type, instructional procedure, 
and the age of the subject in determining degree of learning. 

The present study attempted to implement the extension of con- 
cept learning research in each of the suggested directions. 

Age of subjects . Subjects were fourth- and sixth-grade children. 

Nature of the concepts . Concepts taught were geometric forms 
which had names, some of which were unfamiliar to the children who 
served as subjects. Further, some degree of attribute learning was 
required for the younger children. The concepts studied bore complex 
interrelationships to one another. A strategy for characterizing the 
concepts was developed, which consisted of determining the attributes 
relevant and irrelevant to the concept, and of determining the rela- 
tionship of each concept to the others. 

Measurement of learning . On the basis of logical analysis of the 
nature of a concept, a review of the literature on cognitive processes 
in concept learning, and a review of previously employed concept learn- 
ing measures, eleven tasks were identified which would permit inference 
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of various aspects of concept learning. These tasks were incorporated 
into test items to measure attainment of each concept. 

Instructional procedures . Concepts were taught by a combination 
of definitions and examples. Lessons varied in the number of examples 
presented and the relative emphasis of relevant attribute values. 

The importance of providing a variety of examples has been 
stressed by educational psychologists: "the defining attributes of a 
concept are learned most readily when the concept is encountered in 
a large number of different contexts .... By de-emphasizing the 
particularity of single or homogeneous instances, multicontextual 
learning facilitates the abstraction of commonality, strengthens the 
generality and transferability of the resulting concept, and endows 
it with greater stability [Ausubel, 1968, p. 531]." "A grasp of ab- 
stract concepts, of course, must come from experience with many 
objects. To understand s quareness , for instance, the student must 
see ... a variety of concrete objects, all having the common prop- 
erty of squareness [Stephens, 1956, p. 369]." 

Despite the seemingly obvious truth of the notion that a wide 
variety of instances facilitates concept learning, the research evi- 
dence concerning the effect has been contradictory. Callantine and 
Warren (1955) and Morrisett and Hovland (1959) found better transfer 
to new concept instances when a wide variety of examples were used 
during training. Podell and Carter (1963) found that a large variety 
of instances promoted concept acquisition and resulted in greater 
generalization, especially when the concept was relatively difficult. 
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The results of Gagne and Bassler (1963) indicated significantly lower 
retention for students given only a minimal variety of examples. 

On the other hand, under some circumstances small variety has 
been associated with improved acquisition (Amster, 1966) and generali- 
zation (Amster & Marascuilo, 1965). Remstad (1969) noted little 
effect due to number of instances. A significant interaction between 
number of instances and specific concept was observed by Shore and 
Sechrest (1961). 

Thus, the effect of number of instances has not been clearly 
established. Determination of the particular aspects of concept learn- 
ing affected by this variable may provide clarification. 

Emphasis of relevant attribute values has also been suggested as 
an effective technique of increasing concept attainment (Klausmeier & 
Goodwin, 1966; Ausubel, 1968). The research evidence clearly sup- 
ports this principle. Inducing a set to respond to the relevant dimen- 
sion of a concept by pretraining with words describing that dimension 
facilitated learning performance (Gelfand, 1958). Rasmussen and Archer 
(1961) also noted improved learning when the relevant dimension had 
been labeled during pretraining., Wittrock, Keislar, and Stern (1964) 
discovered that children who were given the label for the relevant 
characteristic of a concept had significantly higher scores than chil- 
dren given more general hints or specific names of instances. Addition 
of a single-word verbal cue drawing attention to the common attribute 
of concept instances greatly increased transfer (Remstad, 1969). 

Thus, the emphasis of relevant attribute values has consistently 
been shown to improve concept learning. It should be noted, however, 
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that in all cases, the concept tasks were inductive in nature. An 
important question is whether this same facilitation would occur if 

the subject were given a definition of the concept, making the task a 
deductive one. Further, it would be of interest to determine what 
aspects of concept learning performance were most affected by this 
variable. 



Purposes and Hypotheses of the Study 

The first objective of this study was to devise a set of proto- 
typic tasks which would test various aspects of concept learning. The 
second objective was to determine the effect of two instructional vari- 
ables on performance of these tasks. 

Two’ hypotheses were tested. The first was that level of concept 
mastery would increase as a function of the number of instances pre- 
sented- Thus, it was predicted that children who had seen eight exam- 
ples of a concept would correctly answer more questions concerning 
that concept than children who had seen only four instances. The 
second hypothesis was that emphasis of relevant attribute values would 
facilitate concept learning. 



Method 

Subjects were 154 fourth-grade and 126 sixth-grade children. 
Eight versions of programed lessons dealing with geometric concepts 
were prepared to vary systematically the independent variables of 
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number of instances and emphasis of relevant attribute values. The 
children studied these lessons approximately 20 minutes a day for four 
days . 

After completion of the lessons, children were given a multiple- 
choice test and a completion test, each consisting of eleven types of 
questions related to concept learning. 

Multivariate analyses of covariance were carried out on total test 
scores and on scores for each type of question in order to determine 
the effects of number of instances and emphasis of relevant attribute 
values. 



Significance of the Study 

The method of analyzing concepts developed for this study may 
be applied to various subject-matter areas. Such a detailed descrip- 
tion of the concepts being learned would permit some control of the 
nature of the concept as a source of variability. The technique of 
measuring concept learning devised for the study provides exact de- 
scription of. the responses from which concept learning may be inferred. 

Specification of the nature of the concept and of the response 
has great utility for both research on instructional variables and 
theory of concept learning. The effect of instructional variations 
may be related to the type of concept and response being considered. 

The differentiation of tasks may clarify the cognitive processes 
entailed in concept learning. 



Chapter II 



REVIEW OF RELATED LITERATURE 



To extend our knowledge about concept, learning, a wider range of 
concepts should be studied. A system for characterizing the essential 
aspects of subject-matter concepts will be described in the first sec- 
tion of this chapter. Utilizing this system, diverse concepts may be 
classified according to their similarities and differences. In the 
second section of the chapter, a strategy for testing concept mastery 
will be derived, based on a review of the literature dealing with 
cognitive processes and testing procedures. Finally, research re- 
lated to the effects of increasing the number of concept examples and 
of emphasizing relevant attribute values will be discussed. 

Analysis of Subject-Matter Concepts 
From its earliest beginnings, experimental study of concept 
learning focused on concepts having clearly specifiable character- 
istics. This trend was crystallized by the publication of an in- 
fluential article by Hovland (1952) which related the amount of in- 
formation conveyed by a concept instance to the number of relevant 
dimensions and values on those dimensions. In order to utilize the 
Hovland informational analysis, the exact number of relevant dimensions 
and values must bo determined. This requirement has tended to 
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restrict experimental material to geometric figures or artificially 
devised stimulus populations. These materials are intended to be 
analogues of naturally occurring concepts. Indeed, Bruner, Goodnow, 
and Austin (1956) used numerous "real-world" examples to demonstrate 
that concepts may be described in terms of criterial attributes and 
conceptual rules. The learning of these "real-world" concepts, how- 
ever, has rarely been studied experimentally. The paucity of such 
experimentation may be due to the fact that the exact number of rele- 
vant attributes and attribute values is sometimes indeterminate in 
naturally occurring concepts. In general, however, some set of 
characteristics common to all examples of a concept may be specified, 
although the set may not be exhaustive. 

It would be desirable to study naturally occurring concepts in 
order to assess the effects of meaningful labels, mode of represen- 
tation, and relationship to other concepts. A report by Berzonsky 
(1968) indicated that analysis of biological concepts in terms of 
defining attributes could be carried out. Due to the nature of the 
concepts, experimental results based on this analysis had direct im- 
plications for both psychology and pedagogy. 

A concept may be described in terms of six characteristics: 

(1) relevant attribute values (properties which are common to all 
instances of the concept) ; (2) concept definition (comprised of the 
relevant attribute values of the concept and the rule combining those 
values); (3) irrelevant attributes (properties which vary from in- 
stance to instance); (4) concept label; (5) supraordinate concepts 
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(more generic concepts having some but not all of the relevant attri- 
bute values of the given concept); and (6) subordinate concepts (more 
specific concepts having all of the relevant attribute values of the 
given concept and other in addition) . 

Although other characteristics might be specified, this set pro- 
vides a basis for comparing concepts which differ in complexity, rule 
difficulty, type of label, and degree of interrelatedness. 

Measurement of Concept Mastery 

Johnson and O'Reilly (1964) have posited that a concept is a 
single hypothetical construct with specifiable relations to different 
observable measures * Although a concept is seen as a single entity, 
Johnson (1964) has suggested that two or more tests be made in order 
to provide convergent evidence for its existence and characteristics. 
Differences which occur between two tests are ascribed by Johnson and 
O'Reilly (1964) to variability in response difficulty, response prac- 
tice, or scoring. The agreement of two tests, however, is not a 
necessary consequence of viewing a concept as a single hypothetical 
construct. The tests may provide evidence concerning different as- 
pects of the concept. Thus, for example, if the subject had cognized 
the relevant attribute values, of a concept but had not cognized the 
label for them, he would be able to classify concept examples but 
unable to verbalize the concept definition. Indeed, a discrepancy 
between the ability to classify and define has been frequently noted 
(Phelan, 1968; Deno, 1968; Remstad, 1969). 
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Although a concept held by a given individual at a given time is 
a single entity, the characteristics of the concept may differ (1) 
among individuals, and (2) in the same individual over time (Klausmeier, 
Sterrett, Frayer, Lewis, Lee, & Bavry, 1969). Therefore, our goal 
should be to characterize the concept held by an individual with 
sufficient precision to detect these differences. 

Several authors have suggested that concept learning is the re- 
sult of complex interplay between elementary cognitive processes. 

Gagne (1968) views classification as an intellectual skill. It is 
one member of a hierarchical set of intellectual skills in which 
subordinate skills provide positive transfer to the learning of super- 
ordinate skills. Gagne cites research, for example, which indicates 
that dimensional discrimination training facilitates classification. 
Other skills which may transfer to concept learning include signal 
learning, stimulus -response learning, learning of verbal associ- 
ates, and multiple discrimination (Gagne, 1965). 

Guilford (1967) has proposed a three-way classification of intel- 
lectual abilities, the s tructure-of-intellect (SI) model. The dimen- 
sions of the model specify the operation, content, and product of a 
given intellectual act. Each factor in the model is defined by 
specifying a level on each of the three dimensions. The processes 
relevant to concept learning may be clarified by identifying them 
with tests of the abilities postulated in Guilford s model. In 
complex learning situations such as a concept-learning task, S does 
not sit passively, learning only at the whim of the experimenter s 
manipulation of conditions . . . S recognizes attributes common to 




the stimuli, he produces and tests hypotheses concerning which 
attributes are relevant, and he remembers what occurred on previous 
trials. The last statement implies all five of the operations of 
the SI model. The potential for the understanding of concept learn- 
ing is in the investigation of these processes that S performs 
between receiving of the stimulus and the production of the overt 
response [Dunham, Guilford, & Hoepfner, 1966, p. 4]." 

Support for the contention that various processes come into play 
during concept formation derives from results indicating that different 
abilities are relevant to different types of concept problems, that 
relevant abilities vary over stages of practice, and differ for solvers 
and non-solvers (Dunham, Guilford, & Hoepfner, 1966; Jones, 1968; 
Manley, 1965). 

Still another theory of the development of classificatory concepts 
has been proposed by Inhelder and Piaget (1959). Seven of the develop- 
mental steps identified were: (1) grouping of two objects on the basis 
of resemblance, (2) grouping of more than two objects, (3) grouping of 
all objects which are in some way alike, (4) grouping regardless of 
the physical proximity of the objects, (5) categorizing the same object 
into more than one group, (6) grouping objects in different ways, and 
(7) forming classes that stand in an inclusive relationship to one 
another. Kofsky (1966) administered tasks designed to measure these 
developmental steps in children 4-9 years of age, and found that 
there was a significant correlation between age and number of task 
successes. Further, the tasks exhibited the predicted order of diffi- 
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Klausmeier (Klausmeier, Harris, Davis, Schwenn, & Frayer, 1968) 
has identified three sets of process configurations involved in con- 
cept learning: (1) analyzing the situation, (2) securing information, 

and (3) processing the information. The information processing steps 
hypothesized by Klausmeier were as follows: 

Sensing external and internal stimulus situations 
Acquiring or manifesting non-labeling responses 
*Associating responses with stimuli along physical dimensions 
Associating sequential S-R associations 

Acquiring labels and associating proper labels with each 
stimulus situation 

Discriminating among many stimuli and responding appropriately 
Cognizing common properties of at least two instances and 
responding with the appropriate single label 
Discriminating the relevant from the irrelevant characteristics 
of the stimulus situation 

Cognizing that two or thiree instances do or do not belong to the 
same set 

Cognizing a relevant attribute or rule by comparing the informa- 
tion presented in positive and negative instances 
Hypothesizing relevant attributes 
Remembering attributes, rules, and hypotheses 
Evaluating relations among attributes or values and rules in 
terms of an hypothesis 

Inferring the concept by inductively arriving at the common 

defining properties and rules; by cognizing logical rela- 
tions among properties and rules [Klausmeier, Harris, 

Davis, Schwenn, & Frayer, 1968, p. 7 ] . 

Drawing upon the work of Gagne, Guilford, Piaget, and Klausmeier, 
the author has postulated the following processes as related to concept 
learning: (1) cognition of the attribute values of concept instances, 

(2) association of attribute values with their labels, (3) cognition 
of an instance as an example or non-example of the concept, (4) asso- 
ciation of a concept instance with its label, (5) cognition of the 
characteristics common to all concept examples. (6) cognition of the 
conceptual rule relating the common characteristics, and (7) cognition 
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of the relationship between the concept and concepts subordinate and 
supraordinate to it. Tests by which the occurrence of these processes 
may be inferred have been developed for use in the present study and 
will be detailed in Chapter III. 

Development of tests related to each of these aspects of concept 
learning may permit detection of specific instructional effects. It 
is probable, of course, that there will be a high correlation between 
measures. Therefore, a strong differential effect would be necessary 
for differences to be apparent. Johnson and Stratton (1966), for 
example, failed to detect specific relationships between training 
method (classifying examples, defining the concept, using it in a 
sentence, and giving synonyms), and tests of comparable behaviors. 

On the other hand, Nuthall (1968) found that comparison of concepts 
during instruction had a detrimental effect on the recognition of 
positive instances and identification of the relationship between the 
concept taught and other concepts. The recognition of negative in- 
stances, however, was facilitated. Thus, it appears that differen- 
tial effects of instruction may in fact occur. 

Number of Instances 

One of the variables examined in the present study was number 
of instances. Since no instances were repeated, an increase in num- 
ber of instances implied an increase in variety of instances. 

Several of the experiments dealing with number of instances have 
related to the formation of a "learning set," rapid solution of a new 
problem of a specified type following practice on a large number of 
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problems having the same general solution. In these studies the 
problems which are presented vary in attributes but have a common 
rule (often relational) joining the attributes. The goal, then, is 
rule learning. 

Adams (1954) presented examples of four concepts consisting of 
horizontal or vertical arrangements of two different attributes. One 
group of subjects received 24 different examples of each concept 
while another group received repeated presentations of a single ex- 
ample of each concept. Both groups then received three presentations 
of a transfer problem. The group trained with a single example of 
each concept was clearly superior on the transfer problem. 

In contrast to Adams' finding that single-example training led 
to more positive transfer, Callantine and Warren (1955) found that 
multiple-example training produced greater transfer. Six groups were 
compared, having 20, 10, 4, 2, 1, or 0 (control group) different 
examples per concept. During training, the group having only one 
example per concept committed the fewest errors. On the transfer 
task, however, the group having 20 examples per concept was signifi- 
cantly superior. Callantine and Warren suggest that due to rapid 
pacing, a non-crrrection procedure, and difficult stimulus materials, 
Adams' subjects who were trained on multiple examples never mastered 
the concepts prior to the transfer task. 

Replicating Adams' procedure, Morrisett and Hovland (1959) 
compared performance of subjects having 1, 3, or 24 examples per 
concept. Subjects having three examples performed at the highest 
level on the transfer test. Results support Callantine and Warren's 



interpretation that the poor performance of Adams' multiple-example 
group was due to low level of learning of each concept. Greater 
transfer, then, is associated with learning the conceptual rule and 
generalizing it to a variety of contexts. In the present study, 
where the conceptual rule is a simple conjunction, one would expect 
that generalization would be facilitated by increasing the variety of 
instances . 

Although the task employed was not a typical concept learning 
task, results of an experiment by Gagne and Bassler (1963) indicate 
greater retention due to variety of task examples. Gagrii and Bassler 
taught concepts in nonmetric geometry to sixth-grade children by means 
of a self -instructional program. The concepts taught were related to 
each other in such a fashion that knowledge of subordinate concepts 
facilitated acquisition of higher-level concepts. One of the variables 
examined in the study was variety of examples presented for each sub- 
ordinate concept. No differences were observed in retention of the 
subordinate concepts over a nine-week period due to variety of exam- 
ples. Retention of the highest-level concept was significantly 
inferior for those students who had a minimal variety of examples of 
the subordinate concepts. 

In the present study, variety of attribute examples and variety of 
concept examples were inversely related. Students who had a larger vari- 
ety of attribute examples had a smaller variety of concept examples. 

Since attribute examples are analogous to the subordinate task examples 
in Gagne and Bassler's study, a wide variety of attribute examples 
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would be expected to facilitate retention of concepts. The net effect 
of increasing the variety of concept examples, therefore, would de- 
pend on the relative importance of variety of attribute examples and 
variety of concept examples. 

Stern (1965) predicted that training with a large number of in- 
stances of a few concepts would result in greater transfer to new 
instances of those concepts. On the other hand, training with a large 
number of concepts would result in greater transfer to new concepts. 

To test these predictions she presented kindergarten and first-grade 
children with 12 instances of two concepts, six instances of four con- 
cepts, or three instances of eight concepts. Contrary to prediction, 
children trained with six instances of four concepts (intermediate 
variety of both instances and concepts) showed greatest transfer to 
both new instances and new concepts. 

A series of experiments have been carried out by Amster [Podell] 
investigating the interaction of variety of instances with age and 
cognitive set. The hypothesis underlying these studies is that under 
an intentional set to learn, deductive strategies will predominate. 

In these circumstances, a large variety of examples would facilitate 
learning by permitting false hypotheses to be rejected in fewer trials 
than a small variety. Under an incidentrl set to learn, no difference 
due to variety is predicted since a large variety might permit more 
efficient summation of correct associations, but a small variety 
would permit better recall of instances. Under intentional set, 
older children are expected to benefit from a large variety of 
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instances. For younger children, no corresponding facilitation is 
expected since the deductive reasoning process is not well-developed 
(Amster, 1966). 

Although Amster has studied the interactions of variety, age, 
and set, only the effect of variety under intentional set for older 
children and adults will be summarized. Podell (1958) presented col- 
lege students with examples of a figural concept having six relevant 
attribute values. Half of the subjects received 12 examples (large 
variety), the other half two examples (small variety). Under inten- 
tional set, the subjects who had seen 12 examples were able to iden- 
tify significantly more relevant attribute values than subjects who 
had seen only two examples. On the other hand, subjects receiving 
the small variety recalled significantly more irrelevant features. 

Podell and Carter (1963) taught verbal concepts to fourth- and 
fifth-grade children by presenting sentences in which the word whose 
meaning was to be learned was replaced by a nonsense word. Four con- 
cepts were taught: two nouns intended to be easy, and two verbs in- 
tended to be difficult- In the large variety condition, each concept 
was presented in the context of six different sentences. In the small 
variety condition, each concept was presented in three different sen- 
tences. For the easy concepts, no significant effects were observed 
as a function of variety. For difficult concepts, however, subjects 
who had received the large variety of instances were better able to 
define the concept. 

The two studies just mentioned found differences in favor of a 
large variety of instances. Later experimentation (Amster & Marascuilo, 
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1965; Amster, 1966) has failed to confirm this advantage. Amster and 
Marascuilo (1965) taught the concept of set union and set intersection 
to fourth-grade children using either a large or small variety of fig- 
ural instances as examples. No significant differences were noted in 
concept learning due to variety of instances. On a generalization 
test which employed words or letters as instances of the concepts, 
subjects who learned the concepts with a small variety of instances 
performed significantly better. Amster and Marascuilo interpret the 
greater generalization to be indicative of a higher degree of learn- 
ing for subjects receiving a small variety of instances. No effect 
of variety was observed by Amster (1966) when fourth- and fifth-grade 
children were taught concepts based on complex aggregates of features 
contained in flags . The dependent variable in this case was accuracy 
of constructing a new flag to exemplify the concept. 

The effect of number of instances may also differ depending on 
the specific concept being learned. Shore and Sechrest (1961) pre- 
sented college students with 3, 9, or 18 different instances for each 
of four concepts based on common characteristics of nonsense syllables. 
The concepts were chosen to represent four levels of difficulty based 
on results of a preliminary study. With recognition of new instances 
as the dependent variable, a significant interaction was obtained. 

When concepts were based on obvious characteristics, a wide variety of 
instances resulted in better performance. When concepts were based on 
less obvious characteristics or required mental transformation of stim- 
uli, a small variety of instances was preferable. 
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In a study which applied response surface methodology to the 
optimization of concept learning performance, Rems tad (1969) carried 
out a sequence of experiments varying a combination of independent 
variables. One of these independent variables was number of instances, 

6 or 9. Although response surface methodology does not test hypotheses 
concerning the effect of individual variables, little change in response 
was noted when the number of instances was increased from 6 to 9. 

In summary, increasing the number of concept instances lias not 
always resulted in improved concept learning performance. There is 
evidence, however, that greater rule learning occurs with a greater 
number of instances. Definition and verbalization of relevant attri- 
bute values may be higher following training with a wide variety of 
instances. In addition, presenting more instances may increase per- 
formance when the common characteristic is an obvious one. 

Emphasis of Relevant Attribute Values 
All instances of a concept share certain common properties. 

These common properties may be called the relevant attribute values 
of the concept. Instances of a particular concept also differ from 
one another in certain ways. These properties which differ from in- 
stance to instance are not relevant to determining the concept and 
are therefore called irrelevant attributes. 

It is evident that any factor which directs attention to the com- 
mon characteristics of the instances may be expected to speed concept 
learning and increase reliability of instance identification. Factors 
which may increase the noticeability of relevant attribute values have 




been summarized by Trabasso and Bower (1968). These factors have been 
classified as: (1) innate (differential sensitivity of the subject to 
cues); (2) stimulus -bound (perceptual arrangements emphasizing cues); 
and (3) past-training (discriminative history and verbal instructions). 
In the present study, verbal cues were employed which focused attention 
on the relevant attributes. This would be classified by Trabasso and 
Bower as a past -training factor. 

Verbal cues have been shown consistently to facilitate concept 
learning. Gelfand (1958) taught different word lists to three groups 
of college students prior to a concept identification task. The lists 
contained words describing relevant dimensions of the concept, words 
describing irrelevant dimensions, or words unrelated to the concept. 

The concept learning task required the subject to sort instances of & 
geometric concept having two relevant dimensions and zero, two, or 
four irrelevant dimensions. Mean errors to solution were significantly 
less for subjects who had previously learned the list of words describ- 
ing the relevant dimensions. The greatest facilitation was noted for 
problems having four irrelevant dimensions. 

A more direct procedure was used by Rasmussen and Archer (1961) 
to direct attention to the relevant dimension of a concept. Subjects 
were assigned to one of two pre training groups: language pretraining 
in which two paralogs were associated with two nonsense shapes , or 
aesthetic pretraining in which pleasantness judgments were elicited 
for the two nonsense shapes. Contrary to expectation, the group 
given aesthetic pretraining performed better than the group given 
language pretraining. Rasmussen and Archer suggested that the 
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aesthetic pretraining may have led subjects to attend to and discri- 
minate among the dimensions of the stimuli. Nevertheless, for subjects 
given language pretraining, performance was significantly better when 
the dimension labeled during pre training was relevant than when it was 
irrelevant. The latter finding lends further support to the facilita- 
tory effect of directing attention to relevant attribute values by 
verbal cues . 

During a three-month period, Wittrock, Keislar, and Stern (1964) 
taught kindergarten children a hierarchy of associations of words 
which were later used as cues in a concept identification experiment. 
The word article (general cue) was associated with the words la. and le^ 
(class cues) which were in turn associated with the Frencn names (spe- 
cific cues) for twelve objects and animals. After the preliminary 
training period, children were given a task requiring them to select 
one of two pictures which went with a model picture. The basis for 
matching was the gender of the French name of the model picture. Chil- 
dren were assigned to four treatment groups which differed in the type 
of cue given during a training task: no cue, general cue, class cue, 
or specific cue. Following the training task, children wci.e given a 
retention test and another transfer test. Children who received the 
class cue during training performed significantly better on immediate 
transfer, delayed transfer, and retention tests than children receiv- 
ing more general or more specific cues . 

The effect of preexperimcntally-learned general, class, and spe- 
cific cues was examined by Wittrock and Keislar (1.96.)). Second- and 
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third-grade children were taught geometric concepts. During instruc- 
tion they were presented with a general cue ("color or shape or some 
other way"), a class cue (color, size, shape, or number), or specific 
cue (e.g. red, black, orange). Note that, unlike the cues used in the 
Wittrock, Keislar , and Stern (1964) experiment, these cues and their 
relationship to one another were known to the children prior to the 
experiment. Learning, retention, and transfer to new instances were 
better for children who received the class or specific cue than for 
children who received the general cue. 

One of the variables included in Remstad's (1969) study of con- 
cept learning optimization was presence or absence of a verbal cue 
intended to draw attention to the relevant attribute. Examples of 
concepts such as quadrilateral and trapezoid were accompanied either 
by the concept name or by the concept name plus a one word cue (e.g. 
for quadrilateral , "count;" for trapezoid , "opposite"). The addition 
of the single word cue produced one of the largest increases in mean 
correct instance identifications of any individual variable. 

In summary, emphasis of relevant attribute values by verbal cues 
has been shown to improve immediate concept learning performance, 
transfer, and retention. 




Chapter III 

DEVELOPMENT OF INSTRUCTIONAL MATERIALS AND TESTS 

Three factors led to the selection of geometric concepts as the 
subject matter of this experiment. First, the logical structure of 
geometry permitted explicit specification of relevant and irrelevant 
attributes of the concepts. Second, geometric concepts could be suit- 
ably presented by textual materials, allowing control of the teacher 
variable. Finally, similarity of these concepts to those frequently 
used in laboratory concept identification experiments admitted the 
possibility of comparison with laboratory research results. 

Since the effect of instructional variations on concept learning 
by both fourth- and sixth-grade children was to be studied, it was 
necessary to select concepts of difficulty appropriate to both grade 
levels. Inspection of elementary mathematics texts suggested that 
quadrilateral , trapezoid , parallelogram , rectangle , rhombus , and 
square are concepts which can be learned by children in fourth grade 
but are seldom completely mastered by sixth graders. Also, examples 
of these six concepts are comprised of the same attributes, making it 
feasible to present the necessary prerequisite knowledge in a brief 
period of time. The concept of kite 1 which is not usually taught in 

1. Use of the concept kite was suggested by Dr. Thomas A. Romberg. 
The definition of kite also was provided by Dr. Romberg. 
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elementary geometry was also included, since its examples are made up 
of the same attributes as the other selected concepts. 

Test Construction 

Each of the selected concepts, quadrilateral , trapezoid , kite , 
parallelogram , rectangle , rhombus , and square , was analyzed by deter- 
mining relevant and irrelevant attributes, definition, exemplars and 
nonexemplars, subordinate and supraordinate concepts. Table 1 lists 
the attributes and attribute values of the concepts. 

TABLE 1 

Attributes and Attribute Values Comprising Examples of the Concepts 
of Quadrilateral, Trapezoid, Kite, Parallelogram, Rectangle, 

Rhombus, and Square 



Attribute 



Attribute Values 



Closed vs. Open Figure 
Dimensionality of Figure 
Number of Sides 
Simple vs. Non-Simple Figure 
Parallelness of Sides 

Relative Length of Sides 

Size of Angles 

Size of Figure 
Orientation of Figure 
Color of Figure 



Closed, Open 

Plane, Solid 

Three, Four, Five, etc. 

Simple, Non-Simple 

Only one pair of parallel sides, 
Two pair of parallel sides 
(opposite sides equal) 

Two pairs of adjacent sides of 
equal length, All four sides 
of equal length 

Right angles, Acute angles, 
Obtuse angles 

Large, Small, etc. 

Horizontal, Tipped, etc. 

Black, Red, etc. 
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Figure. 1 indicates the relationship between the concepts. Relevant 
attribute values for each concept are indicated in parentheses. Lines 
indicate supraordinate-sub ordinate relationships between the concepts, 
with the subordinate concept being placed lower in the figure than the 
supraordinate . Note that in each case, subordinate concepts have all 
of the relevant attribute values of the supraordinate, and others in 
addition. 



QUADRILATERAL 

(closed, plane, four sides, simple) 



PARALLELOGRAM 
(closed, plane, four 
sides, simple, two 
pair of parallel 
sides) 



RECTANGLE 

(closed, plane, four 
sides, simple, 
right angles) 



TRAPEZOID 

(closed, plane, four 
sides, simple, one 
pair of parallel 
sides 



RHOMBUS 
(closed, plane, four 
sides, simple, all 
four sides of equal 
length) 



'KITE 

(closed, plane, four 
sides, simple, two 
pair of adjacent 
sides of equal length) 



SQUARE 

(closed, plane, four 
sides, simple, all 
four sides of equal 
length, right angles) 



Figure 1. Relationships between the concepts of quadrilateral, parallel- 
ogram, trapezoid, kite, rectangle, rhombus, and square. Relevant 
attribute es of each concept are indicated in parentheses. 



A set of behavioral objectives related to concept learning was 
developed for use in this study. These objectives were based on the 
analysis of cognitive processes in concept learning which was detailed 
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in Chapter II. The objectives which were developed are listed in the 
left columns of Tables 2 and 3. Test items exemplifying the behavioral 
objectives related to the concept quadrilateral are given in the right 
columns of Tables 2 and 3. Objectives la- 11a in Table 2 require the 
selection of a response from several alternatives. Objectives lb- lib 
in Table 3 are parallel to the objectives in Table 2 but require the 
production of a response. 

Although specifically developed for this study, the behavioral 
objectives were intended to reflect cognitive processes in concept 
learning regardless of subject-matter content. Examples of the 
application of these same behavioral objectives to transformational 
grammar concepts may be found in a recent report by Frayer, Fredrick, 
and Klausmeier (1969). 

Selection test item types la and 2a require cognition of the 
attribute value being tested and association of the attribute value 
with its label. Item types 3a, 4a, and 5a require cognition of an 
instance and association of the concept with its label. Item types 
6a and 7a require discrimination of the relevant from the irrelevant 
attributes of a concept and association of those attributes with 
their labels. Item types 8a and 9a require cognition of the conceptual 
rule and all relevant attribute values and association of the rule 
and attribute values with their labels. Item types 10a and ila require 
cognition of the relationship between the concept and concepts sub- 
ordinate and supraordinate to it. Production test items are similar 




Behavioral Objectives Related to Concept Learning and Sample 
Selection Test Items for the Concept of Quadrilateral 
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6b. Given the name of a concept, the 6b. How many sides do quadrilaterals have? 

student can supply the names of 
the relevant attribute values of 
the concept. 
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to the selection items, but in each case the process which is being 
tested is what Guilford (1967) would term "convergent production." 

The behavioral objectives may be to some extent hierarchical. 

For example, it seems unlikely that a subject could supply a defi- 
nition of a concept without being able to identify some of its 
relevant attribute values. On the other hand, he could correctly 
identify some of the relevant attribute values, yet misclassify a 
concept instance. On a logical basis, however, we might predict 
that item difficulty would generally increase f. om item type 1 to 
item type 11. Five specific difficulty levels were anticipated: 

Level 1, item types 1 and 2; Level 2, item types 3, 4, and 5; Level 3, 
item types 6 and 7; Level 4, item types 8 and 9; and Level 5, item 
types 10 and 11. 

In order to test the effect of instructional variations on 
attainment of geometric concepts, the behavioral objectives were 
utilized to construct two parallel forms of a multiple-choice test, 
with each form comprised of items as indicated in Table 4. In addi- 
tion, a completion test was constructed, consisting of questions 
parallel to those on the multiple-choice tests but requiring production 
of the answers. The sequence of items in the multiple-choice and 
completion tests was the same. In general, the pattern was to ask 
one question of each item type 1-11, then repeat the cycle until 
the questions of each item type for all concepts were completed. 

The sequence of item types differed for each concept. 
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TABLE 4 







Content 


of Tests 


by Item Type and 


Concept 














Concept 








Item 


Quadri- 






Parallel- 


Rec- 






Total 


Type 


lateral 


Kite 


Trapezoid 


ogram 


tangle 


Rhombus 


Square 


Items 


1 


4 


1 


1 


1 


1 


1 


1 


10 


2 


4 


1 


1 


1 


1 


1 


1 


10 


3 


1 


1 


1 


1 


1 


1 


1 


7 


4 


1 


1 


1 


1 


1 


1 


1 


7 


5 


1 


1 


1 


1 


1 


1 


1 


7 


6 


1 


1 


1 


1 


1 


1 


1 


7 


7 


1 


1 


1 


1 


1 


1 


1 


7 
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1 


1 


1 


1 


1 


1 


1 


7 


9 


1 


1 


1 


1 


1 


1 


1 


7 


10 


1 


1 
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1 
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11 


1 
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0 


J> 
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Lesson Construction 

Lessons were devised to teach the selected geometric concepts. A 
modified linear program format was used to present the lessons. This 
format required the child to respond to questions concerning the ideas 
presented and provided feedback concerning the correctness of his 
answers. Since the child could work independently with this type of 
instructional material, the effect of teacher variability was minimized. 
An attempt was made to include all background information required to 
learn the concepts in order to attenuate the effect of differences in 
previous geometry training. 

Four lessons were developed, each requiring an estimated thirty 
minutes for completion. Lesson content was as follows: 



1 
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Lesson I point, line segment, line, ray, angle 

Lesson II right angle, closed curve, simple curve, plane, 

polygon, parallel, adjacent, opposite, equal length 
Lesson III quadrilateral, kite, trapezoid, parallelogram, 
rectangle, rhombus, square 

Lesson IV relationships between the concepts included in 
Lesson III 

After the lessons were written, a professor and two graduate 
students in mathematics education reviewed them to assure accuracy 
of content. One fourth-grade and one fifth-grade student worked 
through the lessons on an individual basis with the experimenter. 
This procedure disclosed several ambiguities and some vocabulary 
and format problems. Revisions were carried out on the basis of 



the subject-matter consultants* comments and initial tryout results 



Pilot Study of Lessons and Tests 

Purpose 

The pilot study has as its objectives: 

(1) to determine the appropriateness of lesson format and reading 
level for fourth- and sixth-grade students, 

(2) to determine ’ error rate for each lesson frame, as a basis for 
lesson revision, 

(3) to obtain item statistics and reliability information for the 
multiple-choice test items as a basis for test revision, 

(4) to obtain item statistics and reliability information for the 
completion test items as a basis for test revision, and 

(5) to determine time requirements and revise directions. 




Procedure 



Subjects . Lessons and tests were administered to 140 fourth- 
grade and 140 sixth-grade students. The fourth-grade children com- 
prised the entire fourth-grade population of an elementary school in 
a Midwestern suburban community. The sixth-grade children comprised 
the population of 5 sixth-grade classrooms in a nrddle school in the 
same community. 

Treatment . The design of the pilot study is shown in Table 5. 
Four lessons were administered, one per day for four consecutive 
school days. On the school day immediately following the administra- 
tion of the fourth lesson, the multiple-choice test was given. Chil- 
dten at each grade level were randomly assigned to one of the two 
parallel forms, with half of the children taking on°. form, the other 
half the other form. 

Eleven days after administration of the multiple-choice test 
(a school vacation intervened), the completion test was given. All 
children received the same form of this test. 

Results . The distributions of total errors and lesson time re- 
quirements were determined for each lesson at each grade level. 
Descriptive statistics for total errors and working times of lessons 
are presented in Table 6. Mean total error rate varied from 47, to 
11%. All students were able to finish each lesson within a 45-minute 
class period. 

In general, frame error rate was low. Any frame with an error 
rate higher than 157, at either grade level was revised prior to the 



main experiment. 
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TABLE 5 

Design of the Pilot Study 



Day 1 
Day 2 
Day 3 
Day 4 
Day 5 

Day 6 



Grade 4 

Lesson I 
Lesson II 
Lesson III 
Lesson IV 



Multiple-Choice 
Test 
Form AA 



Multiple-Choice 
Test 
Form BB 



Completion Test 



Grade 6 

Lesson I 
Lesson II 
Lesson III 
Lesson IV 



Multiple-Choice 
Test 
Form AA 



Multiple-Choice 
Test 
Form BB 



Completion Test 



TABLE 6 

Means and Standard Deviations of Total Errors and Mean Working Times 
for Pilot Scudy Lessons for Grades 4 and 6 

















Grade 


4 


Grade 6 


Lesson 


Number of 
Frames 


Total 


Errors 


Times 
(min. ) 


Total 


Errors 


Times 
(min. ) 






M 


SD 


M 


M 


SD 


M 


I 


43 


4.9 


3.4 


23.1 


3.5 


2.6 


20.2 


II 


73 


5.7 


5.7 


21.3 


4.8 


3.6 


19.5 


III 


112 


7.8 


6.5 


21.1 


6.5 


5.7 


19.0 


IV 


64 


3.6 


3.6 


17.0 


2.6 


2.3 


14.9 




Summary statistics for the multiple-choice and completion tests are 
given in Tables 7, 8, and 9. All tests were item analyzed by the 
FORTAP computer program (Baker & Martin, 1968). Separate analyses 
were carried out for each test form at each grade level. Scores 
were obtained for the total tests and each of the item type subtests. 

Reliabilities for the total tests at both grade levels were 
high, .85 or greater. Subtest reliabilities varied substantially 
among, different subtests and between grades. In general, reliabili- 
ties were lowest for grade 4 on the higher subtests. Low reliabili- 
ties were anticipated for the subtests since the number of items per 
subtest was small and the items unrefined. 

Item difficulties also differed among subtests and between 
grades, with difficulty being greater for the higher subtests and 
for fourth-grade children. In all cases, mean scores indicated 
better than chance performance. Overall test difficulty was more 
suitable for grade 6 than for grade 4. 

Sixth-grade mean total scores on both forms of the multiple- 
choice test were approximately midway between the expected chance pro- 
portion (257 0 ) and 100%. The mean total score for sixth graders on 
the completion test was also mid-range. 

On the basis of item analyses, individual items were revised to 
eliminate unused distractors, adjust difficulty level, and maximize 
discrimination. Revision of items was carried out with the goal of 
maximizing reliabilities for the item type subtests. 
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Students were to be permitted as much time as they wished to 
complete the tests. During the pilot study, it became evident that 
the completion test would require two entire class periods instead 
of one. Scheduling conflicts prevented allowing this much time. As 
a consequence, many fourth graders and a few sixth graders did not 
complete this test. Scores are therefore lower than they would have 
been had adequate time been available. For the main experiment, the 
test was divided into two parts to be administered on separate days. 



Chapter IV 



METHOD 



The primary purpose of this experiment was to determine the effect 
on concept learning of two instructional variables, number of instances 
and emphasis of relevant attribute values. On the basis of related re- 
search and logical analysis, two predictions were made regarding the 
effect of these variables: (1) level of concept mastery would increase 

as a function of the number of instances presented, and (2) emphasis of 
relevant attribute values would facilitate concept learning. 

Subjects 

The subjects in this study were 154 fourth-grade and 126 sixth- 
grade children. The fourth-grade children constituted the entire fourth- 
grade population of an elementary school located in the same community 
in which the pilot study was conducted. The sixth-grade children com- 
prised the population of five classrooms in the same middle school in 
which the pilot study was carried out. The five classrooms of children 
were selected on the basis of convenience of scheduling from ten hetero- 
geneously grouped classrooms. Two low ability classes and one high 
ability class were deliberately excluded from the study. The study 
began with 169 fourth-grade and 141 sixth-grade children. Fifteen 
students at each grade level were lost due to absences during the 
experiment. 
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The experimenters were one male and one female graduate student, 
both of whom became familiar with procedures and materials prior to 
the experiment. 



Instructional Materials 

To study the effect of number of instances and emphasis of relevant 
attribute values on concept learning, lessons were desired which were 
similar to those normally used in the school, but which rigorously con- 
trolled the variables of interest. Based on results of the pilot study, 
lessons were constructed to attain the twin objectives of realism and 
control. Lessons were designed to be administered on four successive 
school days. Lesson I was the same for all children. Variations in 
Lessons II, III, and IV constituted the experimental treatments. Content 
of lessons for each treatment group is outlined in Table 10. 

TABLE 10 



Content of Instructional Treatments 



Treatment 


Lesson 


I 


II 


III 


IV 


A 

1 


Background 


Attributes 1 


Attributes 2 


Concepts 1 


JL 

B 


Background 


Attributes 1 


Attributes 2 


Concepts 2 


A 


Background 


Attributes 1 


Attributes 2 


Concepts 1 


2 








(Emphasis) 


B 


Background 


Attributes 1 


Attributes 2 


Concepts 2 










(Emphasis ) 


A 

*5 


Background 


Attributes 1 


Concepts 1 


Concepts 2 


B 


Background 


Attributes 2 


Concepts 1 


Concepts 2 


A 


Background 


Attributes 1 


Concepts 1 


Concepts 2 


4 






(Emphasis) 


(Emphasis) 


B 


Background 


Attributes 2 


Concepts 1 


Concepts 2 








(Emphasis) 


(Emphasis) 
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Descriptions of 
Background 

Attributes 
Attributes 
Concepts 1 



Concepts 2 



Concepts 1 
(Emphasis) 



the lessons are as follows: 

Point , line segment , line , ray , and angle were 
introduced. 

1 Right angle , closed curve , simple curve , plane , 
polygon , parallel , adjacent , opposite , and 
equal length were described and examples given. 

2 Content was similar to Attributes 1, with re- 
phrasing of descriptions and different examples 
of the attributes. 

Concepts of quadrilateral , kite , trapezoid , 
parallelogram , rectangle , rhombus , and square 
were introduced. For each concept, the defi- 
nition, two positive and two negative instances 
were presented. 

Content was similar to Concepts 1. Definitions of 
concepts were repeated, and two positive and two 
negative instances different from those in Concepts 1 
were given. 

Content was precisely the same as that of Concepts 1, 
but in addition had questions designed to direct 
attention to the relevant attribute values of the 
concept instance (e.g. "Does this figure have 4 
sides?") and reviewed the relevant attribute values 
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t the conclusion of each concept's presentation 
(e.g. "How many sides does a quadrilateral have?"). 

Concepts 2 Content was precisely the same as Concepts 2, but 
(Emphasis) 

in addition had attention-directing questions and 
reviewed relevant attribute values. 

Tests 

Multiple-choice test . A single multiple-choice test was constructed 
by selecting the best items from the two parallel multiple-choice tests 
used in the pilot study and revising when necessary to improve item char- 
acteristics. The composition of the test was the same as that indicated 
in Table 4. Selection and revision of items were carried out with the 
following goals: 

(1) to make all items of appropriate difficulty for both fourth- and 
sixth-grade children. 

(2) to produce items which would effectively discriminate at both 
fourth- and sixth-grade levels, 

(3) to eliminate non-functioning distractors, and 

(4) to maximize item-type subtest reliabilities. 

Completion test . A completion test was devised by revision of the 
test utilized in the pilot study. The primary changes made were the 
addition of prompts to verbal questions which had proved too difficult. 
Also, some items which required completion of a drawing were changed 
to make the correct answer more perceptually obvious. The test was 
divided into two parts, so that each part could be administered during 
a regular class period. 




1 
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Procedure 

The schedule for the study was as follows: Days 1-4, administration 

I 

of Lessons I-IV; Day 5, multiple-choice test; Day 6, completion test, 
part 1 (preceded by an interval of 72 hours); and Day 7, completion 
test, part 2. 

On the first day of the experiment, the children were given general 
instructions concerning the purpose of the study and procedures to be 
followed in completing the lessons. A copy of these instructions com- 
prises Appendix A. Children were reminded of the essential points in 
these instructions on Days 2, 3, and 4. 

t 

Prior to the beginning of each lesson, new vocabulary was reviewed. 

A vocabulary list was included in each lesson booklet. The experimenter 
read aloud each word on the list and had the children repeat it after 
him. The list was then repeated in random order, and the children were 
directed to raise their hands when they had found the word which had 
been pronounced. After allowing time for most children to locate the 
word, the experimenter called on one of the children to give the number 
of the word which had been pronounced. Thus, feedback was provided to 
all children so they could be sure they had found the correct word. 

This vocabulary review was an attempt to produce sight recognition of 
the words contained in the lesson. 

While the children completed the lessons, the experimenter proctored 
to be sure directions were followed. No assistance was offered to the 
children other than to fulfill requests for pronunciation of words or 
clarification of procedure. 
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Following the experiment, teachers were requested to complete a 
questionnaire indicating which of the concepts had been studied by 
their class during the school year. In addition, teachers were asked 
to estimate the degree of mastery which their students possessed 
of each concept. A copy of the questionnaire comprises Appendix b. 

Experimental Design 

The two independent variables were number of concept examples 
(4 or 8) and emphasis of relevant attribute values (presence or 
absence of attention-directing and review questions). Factorial 
combination of the levels on these variables resulted in four basic 
treatment groups. In addition, however, treatment groups which re- 
ceived only one concept lesson were counterbalanced so that half of 
the children received one concept lesson, half another. For example, 
for Lesson IV of treatment 1, half of the children received Concepts 1, 
the other half Concepts 2. For treatment groups which received only 
one attribute lesson, similar counterbalancing was carried out so 
that half of thu children received Attributes 1, half Attributes 2. 

The nature of this counterbalancing may be clarified by reference to 
Table 10. Counterbalancing of each of the four treatment groups re- 
sulted in a total of eight different treatments. 

Within each of the six fourth-grade and five sixth-grade classes, 
subjects were randomly assigned to one of the eight treatment groups. 
Thus, subjects were nested within classes. The total design may then 
be characterized as a treatments x blocks design with subjects nested 
within class and treatments crossed with class. A two-way fixed effects 
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analysis of variance model was assumed with the mean square error term 
as the denominator of the F-ratio for both main effects and the inter- 
action. The design of the experiment is illustrated in Table 11. 



TABLE 11 

Design of the Experiment 



I 

Grade 




4 Instances 


8 Instances 


Class 


Non-Emphasis 


Emphasis 


Non-Emphasis 


Emphasis 


A 


B 


A 


B 


A 


B 


A 


B 


4 


1 


s i 

S„ 
















2 


















3 


















4 


















5 


















6 


















6 


1 


















2 


















3 


















4 


















5 
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Chapter V 



RESULTS 



Twenty-four scores were obtained for each subject: the total 

test and eleven subtesl scores of the multiple-choice test, and the 
total test and eleven subtest scores of the completion test. In 
addition, the score for each student on the Paragraph Meaning test of 
the Stanford Achievement Test battery (Kelley, Madden, Gardner, & 
Rudman, 1964) was procured from school records for use as a covariate.. 

In the initial section of this chapter, reliability estimates and 
item statistics will be presented for both the multiple-choice and 
completion tests. Subsequently, the multivariate analyses of covari- 
ance on total scores and on subtest scores will be reported. Descrip- 
tive statistics for the lessons and results of the teacher question- 
naire will comprise the final . section. 



Psychometric Characteristics of Tests 

Sample 

The reliability estimates and item statistics reported in this 
section are based on data for all subjects completing each test during 
"he main experiment. The sample size on which the test statistics arc 
based is somewhat larger Lhan that for the multivariate anaJyses to be 
reported later, since subjects missing any lesson or test were dropped 
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from the multivariate analyses. At the fourth-grade level, 167 
subjects were given the multiple-choice test, 173 the completion 
test. At the sixth-grade level, 142 subjects were given the multiple- 
choice test, 136 the completion test. 

* 

Test Statistics 

Tests were analyzed by the FORTRAN Test Analysis Package (Baker & 
Martin, 1968). Separate statistics were computed at each grade levc?l 
for each of the multiple-choice and completion tests' total tests and 
subtests. The means, standard deviations, ranges, standard errors of- { 
measurement, and Hoyt internal consistency reliabilities (Hoyt, 1941) 
for the multiple-choice total test and subtests are presented in 
Tables 12 and 13; for the completion total test and subtests in Tables 
14 and 15. 



A summary of item characteristics (difficulty, X^q, point biserial 
correlation coefficient, and 3 ) for the correct choices for each of the 
total tests and subtests is presented in Tables lb-23. Utilization of 
this set of item characteristics has been suggested by Allen, Feezel, 
Kauffeld, and Harris (1969). Item difficulty refers to the proportion 
of subjects responding correctly to an item. X 50 is the point on the 
criterion scale, given in standard deviation units, corresponding to 
Lne median of the item characteristic curve and is the point of maximum 
discrimination. The criterion used was the total score on the test be- 
ing considered. The point biscrial correlation coefficient and 3 arc 
indices of the discriminating power of an item. The point biserial 
coefficient relates scores on a given item to seorcs on the total test 




TABLE 12 

Reliability Estimates for Multiple-Choice Total Test and Subtests for Grade 
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of which the item is a part. $ is the reciprocal of the standard 
deviation of the item characteristic curve. The skewed distributions 
of the point biserial correlation coefficients and 3's present prob- 
lems in use of the mean as a measure of central tendency. General 
trends may, however, be revealed by inclusion of these statistics 
(Allen ej: al. , 1969). 

Mean scores for the multiple-choice test presented in Tables 12 
and 13 are generally lower than the comparable scores obtained during 
the pilot study. Two differences between the studies may account for 
this decrease in mean scores. First, item revision may have inadver- 
tently increased difficulty although in most cases this was not an 
intended outcome. Second, one of the lessons given in the pilot study 
but not in the main study dealt with the relationships between the 
concepts being taught. This lesson may have increased performance, 
particularly on item types 10 and 11. 

The reliability estimates for the multiple-choice total test and 
subtests reported in Tables 12 and 13 are also somewhat lower than 
those obtained in the pilot study. Particularly low reliabilities 
were found for scales 3 and 5 - 11 at fourth-grade level, and scales 3, 
4, and 11 at sixth-grade level. The reliability of a set of test 
scores is related to other characteristics of the test. Ebel (1965) 
indicated that the reliability coefficient will usually be greater: 

(1) for longer tests than shorter tests, (2) for tests composed of 
homogeneous items than for tests composed of heterogeneous items, 
and (3) for tests whose items arc of intermediate difficulty than for 
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tests comprised primarily of very aifficult or very easy items, and (4) 
for tests whose items are more discriminating than for tests whose items 
are less discriminating. 

Inspection of Tables 16-19 suggests that a combination of these fac- 
tors rather than a single factor accounts for the low reliabilities. A 
smaller number of items, lower item difficulties, and lower mean point 
biserial correlations appear to be associated with low reliability 
coefficients . 

Mean scores for the completion test (Tables 14 and 15) are gener- 
ally higher than those obtained for the pilot study. Insufficient time 
was available for administration of the completion test to fourth-grade 
children during the pilot study. Thus, it was predictable that mean 
scores for fourth graders would be higher when a longer time allowance 
was provided. Also, the retention interval between l.esson TV and the 
completion test was eleven days for the pilot study, but only four days 
for the main study. 

Problems were encountered with completion item type 7. During the 
pilot study, the questions of this type proved highly difficult and non- 
discriminating for fourth grade children. An example of the wording of 
the questions used during the pilot study for this study was "IIow many 
pair of parallel sides does a quadrilateral have?" This proved misleading 
since any single number (e.g. 0) would be incorrect. In the main study 
these questions were reworded to say, for example, "What is true about 
the number of parallel sides in a quadrilateral?" This wording led to 
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Sequence Effects 

Mean scores on the completion test were lower than mean scores on 
the multiple-choice Lest. Since there were four alternative choices 
for each item on the multiple-choice test, chance performance was 257,. 
Chance performance on the completion tesL was indeterminate. The fact 
that some completion items had strong prompts and thal a supplementary 
word list was provided probably raised chance performance above 07,. 

It is unlikely, however, that it reached 257,. Taking into account 
the differences in chance levels, performance on the two tests may 
have been equivalent. 

Some evidence exists that there is a facilitative sequence effect 
when a subject takes the multiple-choice form of a test prior to the 
completion form. Heim and Watts (1967) administered multiple-choice 
and open-ended forms of a vocabulary test to sailors. Half of the 
subjects took the multiple-choice form first, immediately followed by 
the open-ended form. For the other half of the subjects, the order 
was reversed. The mean scores for the two groups on the multiple- 
choice test were almost identical, indicating that experience gained 
by taking the open-ended test did not facilitate performance on the 
multiple-choice test. On the other hand, mean scores for the two 
groups on the open-ended test differed significantly, with the higher 
mean s^ore being attained by the group which had previously taken the 
multiple-choice form. 

Similar effects were noted by Deno (1968), who required college 
students to classify, define, and generate examples of psychological 
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concepts. Subjects in Group I were given the classification task 
first, while subjects in Group XX were given the definition and 
generation of novel examples tasks first. No differences were ob- 
served between groups on the classification task. Group I was 
significantly better than Group II on definition and generation 
of examples. 

In the present experiment, the completion test was administered 
after the multiple-choice test. Thus, a facilitating sequence effect 
would be anticipated. On the other hand, the two parts of the comple- 
tion test were administered three and four days, respectively, after 
the multiple-choice test. This time interval would be predicted to 
cause a decrement in performance. 

Relationships Among the Subtests 

In Chapter III, it was suggested that item difficulties would be 
expected to increase from item type 1 to item type 11, with five levels 
of difficulty: Level 1, types 1 and 2; Level 2, types 3, 4, and 5; 

Level 3, types 6 and 7; Level 4, types 8 and 9; and Level 5, types 
10 and 11. Inspecting the mean item difficulties (Tables 16 and 17), 
we note that two departures from this order occurred for the multiple- 
choice test. Item type 9 was intermediate in difficulty between item 
types 5 and 6 and was therefore easier than had been predicted. Item 
type 7 was more difficult than predicted, especially for fourth-grade 
children. For the completion test (Tables 20 and 21), three departures 
from prediction occurred. Item type 11 was easier than expected, while 
item types 7 and 9 were harder than expected.. The differences between 
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item type 9 on the multiple-choice test and item type 9 on the completion 
test are especially interesting. Selecting the correct definition for a 
concept is an easier task than anticipated, while supplying the definition 
is a more difficult task. 

Multivariate Analysis of Data 

Various guidelines have been offered concerning minimal acceptable 
reliability coeff icents . Relatively low reliability coefficients may 
still permit accurate conclusions concerning group means. Nevertheless, 
it is c lea*, that no firm conclusions may be drawn from comparison of 
scores on tests with low reliabilities. For this reason, the primary 
analyses were carried out on scores for the total tests rather than 
the subtests. 

All multivariate analyses were carried out using Finn's (1968) 
Multivariance computer program. Separate multivariate analyses of 
covariance were carried out for each grade level. Dependent variables 
were total score on the multiple-choice test (MT) and total score on 
the completion test (CT) . Means and standard deviations of total test 
scores are presented in Table 24. The covariate was the raw score on 
the Paragraph Meaning test of the Stanford Achievement Test battery. 

This was selected as a covariate in order to reduce variability due to 
differences in reading ability. According to the authors (Kelley, 

Madden, Gardner, & Rudman, 1964), the Paragraph Meaning test measures 
the student's ability to comprehend connected discourse involving 
levels of comprehension from recognition to inference. 
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A multivariate regression analysis was carried out to analyze the 
relationship of the covariate to the dependent variables. Tables 25 
and 26 contain the multivariate and univariate statistics summarizing 
the regression analysis. The multivariate analysis reveals that the 
covariate has a highly significant correlation with the dependent 
variables. The univariate 3? * s indicate that a significant amount of 

each dependent variable's variance can be predicted by the covariate. 

2 

Since R equals the percent of variance predicted, we can see that 
the amount of variance predicted ranges from 52% for fourth-grade 
children on the completion test to 12% for sixth-grade children on 
the multiple-choice test. 

The relationship between the two dependent variables and the 
amount of that relationship due to covariation with reading ability 
are suggested by the following comparisons. The correlation between 
MT and CT is .72 for fourth grade, .81 for sixth grade. The partial 
correlations for the same variables after the linear effects of the 
covariate have been removed are .49 for fourth grade, .78 for sixth 
grade. Thus, total scores for the multiple-choice and completion 
tests are. highly related for both fourth- and sixth-grade children. 

For fourth-grade children, however, this relationship is largely due 
to reading skill. 

Since the number of subjects in the cells varied slightly, the 
multivariate analysis of covariance design is non-or ihogonal . Because 
of this, the effects are not independent and are estimated in stepwise 
fashion. The effects of greatest interest are ordered last to obtain 
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unbiased estimates oi' them. Results of the multivariate analysis of 
covariance of total test scores for Grade 4- are presented in iable 27. 
Estimates and tests of effects were carried out in the indicated order. 

The significance level adopted in this experiment for the multi- 
variate analyses was .05. When univariate analyses were interpreted, 
the alpha level was scaled down in order to control the error rate for 
the tests considered jointly. A strategy suggested by Miller (1966) 
was employed, setting the significance level for an individual F at 
a/k, where k is the number of F tests being interpreted. Thus, when 
two F tests were examined, the critical probability level was set at 
.025 in order to maintain an overall error rate of .05. When eleven 
F tests were considered, the probability level was set at .005. 

There was significant variation among the mean vectors over the 
six class groups. Univariate F statistics were computed for each 
variable. Only the univariate JF for the completion test score was 
significant, suggesting that this element of the vector accounted 
for most of the variation in the mean vectors. To explore the pos- 
sibility that the class effect was due to differences between experi- 
menters, a _t test was carried out on the difference between mean scores 
on the completion test for subjects run by each experimenter. The 
pooled within-cell variance was used to obtain an estimate of che 
standard error of the. difference. The experimenter effect was not 
significant (t = .72, df = 106, £ < .5), leading to the conclusion 
that the significant class effect was due to differences among the 
class groups rather than between experimenters. 
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To clarify the nature of the significant class x treatment inter- 
action, the degrees of freedom were partitioned, and separate tests 
were carried out on seven different class x treatment comparisons. 
Results suggest that the interaction is due to differences between 
classes in relative achievement for treatments 4A and 4B. Thus, in 
some classes, achievement is higher for 4A, in others for 4lJ. This 
effect occurs for both the multiple-choice and completion tesLs. 

The effect of primary interest was the highly significant varia- 
tion in mean vectors due to emphasis of relevant attribute values. 

The univariate j?’s for both dependent variables were significant, 
indicating that scores on both the multiple-choice and completion 
tests contributed to the effect. The observed mean score was 35.55 
on the multiple-choice test for subjects whose lessons emphasized 
relevant attribute values and 31.90 for subjects whose lessons did 
not emphasize relevant attribute values. The observed mean score 
on the completion test for emphasis groups was 29.20, for the non- 
emphasis groups, 25.82. 

The effect of number of instances and the interaction between 
number of instances and emphasis of relevant attribute values were 
not significant. 

As was mentioned previously, analyses were carried out on total 
scores since reliabilities for the subtests were low. In order to 
gain a preliminary notion of which variables were affected most by 
the emphasis of relevant attribute values, further analyses were 
carried out using subtest scores as dependent variables. 
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scores for subjects receiving each level of the two independent vari- 
ables are presented in Table 28. Results of the multivariate and 
univariate analyses for the effects of number of instances and empha- 
sis of relevant attribute values are presented in Tables 29 and 30. 

The multivariate F's for the effect of number of instances were 
nonsignificant. The one indication of an effect is found in the sig- 
nificant univariate F for multiple-choice subtest 4. The mean score 
was 3.19 on this subtest for subjects who were presented with four 
examples, and 3.66 for subjects who were presented with eight examples. 

With the eleven multiple-choice subtest scores as dependent vari- 
ables, a significant multivariate F was obtained for the comparison 
between emphasis and non-emphasis groups. Subsequent examination of 
the univariate F's indicated a significant value for multiple-choice 
subtest 2. Subtest 6 approached significance. A significant multi- 
variate F was also obtained fbr the comparison between emphasis and 
non-emphasis groups using the eleven completion subtest scores as 
dependent variables. Among the univariate F's, only that for comple- 
tion test 2 was significant. 

/ 

One must remember, however, that results for all analyses carried 
out on subtest scores must be interpreted with caution due to low 
reliabilities . 

Results of the multivariate analysis of covariance of total test 
scores for Grade 6 are presented in Table. 31. There was no signifi- 
cant variation among the mean vectors for any of the main effects or 
interactions. Mean subtest scores for Grade 6 are reported in Table 
32, and results of multivariate and univariate analyses of subtest 
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TABLE 28 



Observed Mean Subtest Scores for Grade 4 Students for Each 
Level of Emphasis of Attribute Values and Number of Instances 





Subtest 


Number of 
Items 


Standard 
Deviation 
1 — — 


Non -Emphasis 
(N=7 8) 


Emphasis 

(N-76) 




Difference 


4 Instances 
(N=77) 


8 Instances 
(N=77) 


Difference 


M 


i 


10 


1.96 


5.83 


6.39 


+ 


.56 


6.31 


5.91 


-.40 


M 


2 


10 


1.97 


4.53 


5.82 


+1 


.29 


5.21 


5.12 


-.09 


M 


3 


7 


1.28 


3.08 


3.30 


+ 


.22 


3.17 


3.21 


+ . 04 


M 


4 


7 


1.40 


3.36 


3.50 


+ 


.14 


3.19 


3.66 


+.47 


M 


5 


7 


1.28 


2.91 


2.70 


- 


.21 


2.90 


2.71 


-.19 


M 


6 


7 


1.19 


2.12 


2.79 


+ 


.67 


2.47 


2.43 


-.04 


M 


7 


7 


1.07 


1.74 


1.99 


+ 


.25 


1.94 


1.79 


-.15 


M 


8 


7 


1.17 


2.31 


2.43 


+ 


.12 


2.40 


2.34 


-.06 


M 


9 


7 


1.36 


2.56 


2.74 


+ 


.18 


2.71 


2.58 


-.13 


M 


10 


7 


1.25 


2.12 


2.33 


+ 


.21 


2.06 


2.38 


+ .32 


M 


11 


5 


0.94 


1.35 


1.57 


+ 


.22 


1.40 


1.51 


+ .11 


C 


1 


10 


1.93 


5.99 


6.63 


+ 


.64 


6.49 


6.12 


-.37 


C 


2 


10 


2.01 


4.63 


5.58 


+ 


.95 


5.31 


4.88 


-.43 


C 


3 


7 


1.56 


3.50 


3.89 


+ 


.39 


3.77 


3.61 


-.16 


C 


4 


7 


1.61 


2.49 


2.90 


+ 


.41 


2.68 


2.73 


+ .05 


C 


5 


7 


1.35 


2.28 


2.39 


+ 


.11 


2.48 


2.19 


-.29 


C 


6 


7 


1.13 


1.74 


2.13 


+ 


.39 


2.00 


1.87 


-.13 


C 


7 


7 


0.69 


0.97 


1.11 


+ 


.14 


1.05 


1.03 


-.02 


C 


8 


7 


1.06 


1.10 


1.21 


+ 


.11 


1.14 


1.17 


+.03 


C 


9 


7 


0.84 


0.56 


0.62 


+ 


,06 


0 . 66 


0.52 


-.14 


C 


10 


7 


1.02 


1.15 


0.91 


- 


.24 


1.01 


1.05 


+ .04 


C 


11 


5 


1.06 


1.41 


1.82 


+ 


.41 . 


1.45 


1.75 


+ .30 
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Multivariate and Univariate Analyses for Effect of Emphasis of 
Attributes on Subtest Scores for Grade 4 
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TABLE 32 



Observed Mean Subtest Scores for Grade 6 Students for Each Level 
of Emphasis of Relevant Attribute Values and Number of Instances 



Subtest 


Number of 
I terns 


i 

Standard 

Deviation 


Non-Emphasis 

( N = 62 ) 


Emphasis 

( N = 64 ) 


Difference 


4 Instances 
( N ~ 65 ) 


8 Instances 
( N = 61 ) 


Difference 


M 1 


10 


1.60 


7.84 


8.02 


+ . 18 


8.06 


7.82 


-.24 


M 2 


10 


2.04 


6.71 


8.00 


+ 1.29 


7.53 


7.16 


-.37 


M 3 


7 


1.36 


3.69 


3.55 


- .14 


3.72 


3.51 


-.21 


M 4 


7 


1.43 


4.50 


4.37 


- .13 


4.35 


4.52 


+ .17 


M 5 


7 


1.75 


3.45 


3.48 


+ .03 


3.46 


3.48 


+.02 


M 6 


7 


1.51 


3.19 


3.56 


+ .37 


3.25 


3.52 


+ .27 


M 7 


7 


1.61 


2.89 


3.08 


+ .19 


2.94 


3.03 


+ .09 


M 8 


7 


1.46 


2.79 


3.13 


+ . 34 


2.82 


3.11 


+ .29 


M 9 


7 


1.51 


3.56 


3.83 . 


+ .27 


3.66 


3.74 


+ . 08 


M 10 


7 


1.57 


2.74 


3.13 


+ .39 


2.71 


3.18 


+.47 


M 11 


5 


1.06 


1.82 


1.83 


+ .01 


1.77 


1.89 


+ .12 


C 1 


10 


1.68 


7.89 


8.14 


+ .25 


8.02 


8.02 


.00 


C 2 


10 


1.50 


7.02 


7.58 


+ .56 


7.28 


7.33 


+.05 


C 3 


7 


1.29 


4.76 


4.97 


+ .21 


4.86 


4.87 


+ .01 


C 4 


7 


1.75 


3.74 


3.59 


- .15 


3.78 


3.54 


-.24 


C 5 


7 


1.48 


3.21 


3.20 


- .01 


3.37 


3.03 


-.34 


C 6 


7 


1.53 


2.84 


3.33 


+ .49 


2.95 


3.23 


+ .28 


C 7 


7 


0.78 


1.24 


1.31 


+ .07 


1 .28 


1.28 


.00 


C 8 


7 


1.50 


1.77 


2.38 


+ .61 


1.98 


2.18 


+ .20 


C 9 


7 


1.65 


1.94 


1.91 


- .03 


1.85 


2.00 


t - . 1 5 


C 10 


7 


1.31 


1.90 


1.84 


- .06 


1.74 


2.02 


+ .28 


C 11 


5 


1.35 


2.27 


2.20 


- .07 


2.20 


2.28 


+ .08 



T 
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scores in Tables 33 and 34. With the multiple-choice subtest scores 
as dependent variables, a significant multivariate F was obtained for 
the comparison between emphasis and non-emphasis groups. Examination 
of the univariate F's for the multiple-choice subtests revealed a sig- 
nificant value only for item type 2. With completion subtest scores 
as dependent variables, no significant effects were noted. 

Lesson Statistics 

Although theory and practice in programed instruction has in 
general stressed the need for low error rate, recent thinking has been 
to discount error rate as a validating criterion for a program. As 
Lumsdaine (1965) has pointed out, an end-of-program test following a 
retention interval is probably a more accurate measure of program ef- 
fectiveness than error rate since it is free of immediate context 
cues. The primary usefulness of error rate is in suggesting specific 
aspects of a program which may need revision. On the other hand, it 
is unlikely that a program with a very high error rate is an effective 
program. Therefore, error rate may be considered a necessary but not 

sufficient condition for a good program. The error rates for all 

lessons used in the present experiment appeared to be acceptably low 
for subjects at both grade levels. At the fourth-grade level, total 
error rates varied from 2.2% to 12.5%. At the sixth -grade level, total 

error rates ranged from 2.2% to 9.3%. 

Since lessons varied in length, it is important to note the aver- 
age length of time spent in study of the lessons by each treatment 




CO 

CO 

w 

p 



M-M 

O 

U 

0) 

s 

D 

S3 

M-M 

O 

4J 

O 

0) 

M-M 

M-M 

w 

Pi 

o 

LM 

co 

0) 

CO 



Ctf 

§ 5 

H 



0) 

4J 

•H 

Pi 

CXJ 

> 

c 

p 



vO 

0) 

xJ 

d 

Pi 

O 

Pi 

o 

MM 

CO 

0) 

Pi 

o 

o 

CO 

4J 

CO 

<1> 

4J 

P 

2 

CO 

c 

o 

CO 







x: 




























H 


o 


00 


o 


00 


o\ 


r— 1 


CT\ 


00 


o 




vO 












o 


vO 




00 


<J\ 


m 


m 


r*^ 


CM 






CO 


r— 1 


<J\ 


00 


<J\ 




co 


co 


vO 


cn 


r— 1 






00 


CO 






00 


CO 


cm 


CN 


o\ 


CM 


m 


co 








<u 


























v£> 


p 


o 


o 


o 


o 


o 


O 


o 


O 


o 


o 


o 




vO 




























00 • 


CL 
























CO 


CO O 


























4J 




























CO 


^ V 


























<1> 


♦ 


























4J 


O CL 


























P 


II 


























D 


Pm 


Pm 
























CO 


0) 


0) 
























C 


4J 


4J 




CM 


o 


CO 




o 


r*^ 


CM 


G\ 




r— 1 


o 


d 


cO 


r— 1 


o 


CO 


m 


p— 1 


OJ 


in 


m 


<t 


o 






*H m 




r— 1 




CM 


CM 




r— t 


o 


m 


00 


r— 1 


00 


4J 




Pi 


o 




o 




co 


<t 


o 


CM 


CM 


o 


o 


0) 




cO 
























r— 1 


> XJ 


> 


o 


o 


o 


o 


r— 1 


r— 1 


o 


r— 1 


O 


1— < 


o 


CL 




























S 


4J CO 


c 
























o 


r— 1 


p 
























o 


D r-M 

s »— * 



























Pm 

Q 



Pm 

Q 



C 



4J 

CO 

0) 

4J 

P 

D 

CO 



C 

cU 



4J 

CO 

Cl) 

4J 

P 

D 

CO 



m 





<1> 








m 


H 


00 


CM 






co 


co 


r-M 




o 


CM 


00 


Xf 


O 








m 






r— 1 


vO 


o 




o 


r- 


o 


o 


in 


co 


c 


c 




CO 




<t 


CO 


<j\ 


00 


m 


pM 




vO 


00 


r^ 


<t 


00 


CM 


co 


cO 




4J 


co 


co 


CO 


co 




co 


m 


00 




vO 


CM 




o 


m 




4J 




CO 




• 


0) 
























<1> 


CO 




0) 


co 


o 


p 


o 


o 


o 


o 


o 


o 


O 


o 


o 


o 


o 


4J 


C 




4J 


r— 1 




























co 


H 




P 


* 


V 


Pm 






























0 


r— 1 


























< . 

r-i 






CO 


II 


PL 


































Pm 




























> 






0) 




































o 


<1) 




























4J 








4J 




























pM 






O 


cO 




Pm 
























D 






p 






























s 






o 


U 




0) 






























1 


cO 


m 


4J 


co 


vO 


r— 1 


r*H 


CM 


<t 


<t 


r— 1 


<j\ 


<j\ 


cr\ 








<1) 


> 


r*^ 


cO 


CM 


o 


o 




m 


o 


co 


i— 1 


o 


co 


cr\ 








r— 1 


•H 






CM 


o 


vO 


co 


CM 


m 


vO 


co 


pM 


co 


o 








CL 


4J 


xJ 


Pi 


r*^ 


m 


00 




o 


m 


pM 


CM 


pM 


o 


<t 








*H 


rM 


C 


cO 


• 




























4J 


d 


cO 


> 


o 


o 


o 


o 


o 


o 


O 


r— 1 


O 


co 


o 








r— 1 


s 


































D 




rM 


c 






























£ 




i— 1 

II 


p 

























CSJ 


co 




m 


vD 




00 




o 

r— 1 


s 


£ 


£ 


S — 1 








VH 
S H 


w— ♦ 




Degrees of Freedom for Hypothesis = 1 Degrees of Freedom for Hypothesis 
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group (Table 35). In interpreting the data, two facts should be taken 
into account. First, there was fairly wide variability between sub- 
jects in completion times. Second, there was some unreliability in 
the self-reports of time elapsed due to errors in telling time. Never- 
theless, there is at least one clear trend in the data. Subjects 
having lessons emphasizing relevant attribute values spent longer times 
studying the lessons than the other subjects. Fourth-grade students 
having lessons emphasizing relevant attribute values spent an average 
of 70.1 minutes completing the lessons, while students having lessons 
not emphasizing relevant attribute values spent an average of only 
50.0 minutes. For sixth-grade students the comparable times were 
emphasis, 63.4 minutes j nen-emphasis , 46.9 minutes. 

Teacher Questionnaires 

All six fourth-grade teachers and four of the five sixth-grade 
teachers responded to the questionnaire regarding their students 
knowledge of the concepts dealt with in this experiment. At fourth- 
grade level the number of teachers indicating study of various con- 
cepts during the past school year were: equal length , 5; right angle, 

4; rectangle and square , 3; closed curve , simple curve , and opposite, 
1. All of these lessons occurred five months prior to the experiment. 
In addition, one teacher dealt with the concepts of plane figure and 
parallel one week prior to the experiment. The estimates of fourth- 
grade teachers regarding their students' mastery of concepts were 
quite variable. In general, they believed their students were 
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completely unfamiliar with most of the concepts, had some mastery of 
rectangle , square , and equal length , and had not completely mastered 
any of the concepts. 

At sixth-grade level no teachers reported teaching any of the 
concepts during the past school year. They pointed ou»- that geometry 
was not dealt with in the sixth-grade textbook but had been stressed 
in the fifth-grade text. The teachers, on the whole, believed their 
students had some knowledge, but not mastery, of most of the concepts. 
Thus, the assumption that neither fourth- nor sixth-grade children had 
mastered the concepts prior to the experiment was supported by teacher 
judgments . 



o 



Chapter VI 



DISCUSSION AND SUMMARY 

This study had two major objectives: (1) to devise a set of 

prototypic tasks which would test various aspects of concept learning, 
and (2) to determine the effect of two instructional variables, num- 
ber of instances presented and emphasis of relevant attribute values, 
on performance of those tasks. 

Concept Learning Tests 

A set of eleven tasks was developed on the basis of analysis of 
the nature of a concept ,' review of the literature on cognitive pro- 
cesses in concept learning, and review of testing procedures used in 
previous concept learning studies. Each of the basic tasks was then 
differentiated into two parallel tasks, one requiring recognition of 
the correct answer, the other requiring production of the correct 
answer . 

An important question to be answered is whether these tasks are 
equivalent measures of a single mediating construct (Johnson, 1964, 
Johnson & O’Reilly, 1964; Johnson & Stratton, 1966) or whether they 
are distinct, hierarchically-related tasks reflecting different 
aspects of concept learning (Dunham, Guilford, & lloepfnor, 1966; 

Deno, 1968; Gagne, 1968; Klausmeier, Harris, Davis, Schwenn, & 

Prayer, 1968). Both the multiple-choice and completion tests 
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comprised of the eleven tasks showed high internal consistency re- 
liabilities (.81-.87), indicating a high degree of relationship be- 
tween the tasks. The degree to which the subtests based on partic- 
ular tasks may be distinguished from one another may not be deter- 
mined from this study. . A factor analytic study would clarify the 
interrelationship among subtests. The fact that emphasis of rele- 
vant attribute values and increase in number of instances appeared 
to affect performance on some tasks more than others lends support 
to the belief that the tasks may be differentiated. 

Item difficulties generally increased from task levels 1-11. 
Whether this may be regarded as evidence for the existence of a 
hierarchy of task complexity, however, is open to question. Kropp, 
Stoker, and Bashaw (1966) in testing the hierarchical nature of the 
Taxonomy of Educational Objectives: Cognitive Domain (Bloom, 1956) 

hypothesized that means of scores for taxonomy levels would decrease 
as the complexity level increased. With few exceptions, the. pattern 
of means supported the hypothesis. On the other hand, simplex analy- 
sis suggested that the imputed order of complexity of taxonomy levels 
might be incorrect, in which case item difficulty and complexity 
would be discrepant. 

Guttman (1954) asserted that there is no necessary relationship 
between the complexity of intellectual processes and the difficulty 
of items which are intended to measure them. Support for this asser- 
tion was provided by Crawford (1968) who found that the order of 
difficulty was generally different from the order of complexity for 




tests based on the levels of a taxonomy of intellectual processes. 

We may therefore conclude that, although the tasks utilized in this 
experiment may form a hierarchy, mean task difficulties alone are 
not sufficient evidence to verify this structure. Refinement of the 
tests should be carried out and simplex analyses utilized to deter- 
mine order of complexity. 

In Chapter V it was noted that several sub tests had low inter- 
nal consistency reliabilities. The small number of items and wide 
range of item difficulties on these subtests may largely account for 
the low reliabilities. Expansion of these sub tests might increase 
reliabilities and also permit more analytical differentiation of 
levels of concept mastery. 

For example, several items might be written in the multiple- 
choice format of tasks 3 and 4, which require recognition of concept 
examples and non-examples. These items could contrast examples of 
the concept with non-examples differing in the number of relevant 
attribute values which they lack. Such a systematic construction 
of distractors has been suggested by Guttman and Schlesinger (1967). 
The concept is analyzed to determine the number of facets (relevant 
attribute values) which it posesses. Non-examples are classified 
according to the number of facets on which they differ from the 
concept. As the number increases, similarity decreases and the dis- 
tinction between examples and non-examples becomes more obvious. 
Construction of distractors in this manner would also permit detec- 
tion of the particular facets on which the student makes his errors. 
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An alternative method of presenting tasks 3 and 4 in the 
multiple-choice format would be to present the student with an array 
consisting of concept examples and non-examples and have him check 
those which are examples. This would reveal incomplete concepts 
(some examples not checked) and overgeneralization (some non-example. c . 
checked) . 

For the multiple-choice format of task 5, a list of concept 
labels could be provided and the student instructed to check all 
those which are correct. This would be particularly appropriate when 
one desires to test knowledge of more generic labels. Weaver (1966) 
has used this approach in an inventory which tests knowledge of the 
names for geometric figures. The figure is presented, and the stu- 
dent indicates for each of six suggested names whether it is a cor- 
rect name for that figure. More than one name may be assigned to 
each figure. 

For each concept, multiple-choice items of types 6 and 7 could 
be written for each relevant and irrelevant attribute. 

Definitions offered on task 9 of the completion test could be 
assigned weighted scores on the basis of their quality. A possible 
system of scoring definitions has been proposed by Podell (1958) in 
which points are given for each common feature (relevant attribute 
value) and variable feature (irrelevant attribute) mentioned in the 
definition. It should be noted that when variable features are men- 
tioned in a definition, the student should indicate that the feature 
is variable. For example, if "parallel sides" are included in a 
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definition of quadrilateral , it should be specified that a quadrilat- 
eral may or may not have parallel sides. If it were indicated that 
all quadrilaterals had parallel sides, the definition would be too 
restrictive and, therefore, less correct. 

Tabulation of the answers given to completion items may provide 
insights into the nature of the problems encountered by students in 
learning concepts. Interference due to different meanings previously 
associated with concept labels may be indicated by the following 
responses: a simple curve (a curve which can be drawn without cross- 

ing itself at any point) was Called an "easy drawing;" the angles of 
a rectangle were described as "2 right angles and 2 left angles." 

Also, interference may occur due to similarity of concept labels. 
A large number of students stated that a rectangle has three sides. 
Although it is possible that the students believed that rectangles 
have three sides, it is equally plausible that they confused the words 
"rectangle" and "triangle." The latter interpretation was suggested 
by the fact that one student began his definition of rectangle with 

the words "A triangle is . . . ." 

In summary, a test comprised of items written to measure the 
achievement of behavioral objectives related to concept learning 
had high reliability. The degree to which each objective reflects a 
different ability c ho way in which these abilities are related to 
one another should be further explored by factor and simplex analyses. 
The fact that performances on subtests were differentially affected 
by instructional variations suggests that the subtests may be measur- 
ing somewhat different abilities. 
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Number of Instances 

For fourth-grade students, number of concept instances had no 
significant effect on total test scores. Further no significant 
effects were noted for the multiple-choice or completion subtests 
considered multivariately. The only indication of an effect was on 
multiple-choice subtest 4. Students who received eight instances per- 
formed significantly better than students who received only four 
instances. Thus, the larger number of instances facilitated recog- 
nition of non-examples. 

At sixth-grade level, no effects were noted for total test 
scores or any of the subtest scores. 

In interpreting the general lack of effect of this variable, one 
should observe that all groups received both positive and negative 
instances of the concepts. Greater effects might have resulted from 
use of positive instances only. In addition, four instances provide 
a fairly large variety of examples. It is probable that the effect 
of four versus zero or one instance would be greater than the effect 
of four versus eight instances. 

Emphasis of Relevant Attribute Values 

For fourth-grade students, emphasis of relevant attribute values 
had significant effects both on total test scores and on the multiple- 
choice and completion subtest scores considered multivariately. 
Univariate F's for multiple-choice subtest 2 and completion subtest 2 
were significant. 
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At sixth-grade level, multivariate F's for total test scores and 
for scores on the completion subtests were not significant. The multi- 
variate F for multiple-choice subtest scores indicated the presence 
of an effect, which appeared to be due in large part to subtest 2. 

Taken together, these results suggest that the greatest effect 
of emphasis of relevant attribute values is on the ability to correctly 
label attribute values. 

Summary 

Eight versions of programed lessons dealing with geometric con- 
cepts were developed which systematically varied the independent 
variables of number of instances and emphasis of relevant attribute 
values. Two hypotheses were tested: (1) level of concept mastery 

would increase as a function of number of instances presented, and 
(2) emphasis of relevant attribute values would facilitate concept 
learning. 

Ss were 154 fourth-grade and 126 sixth-grade children. Students 
at each grade level were randomly assigned to the eight treatment 
conditions. The basic design of the experiment was a 2 x 2 factorial 
design with two levels of number of instances (4 or 8) and two levels 
of emphas.t.s of relevant attribute values (presence or absence of 
attention-directing and review questions) . Counterbalancing of 
specific lesson content in the four treatment groups of the 2x2 
design resulted in the eight treatment conditions. 
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Ss studied the lessons approximately 20 minutes per day for four 
days. After studying the lessons, children were given a multiple- 
choice test and a completion test, each consisting of eleven types of 
questions related to concept learning. 

The essential findings of the study were: 

(1) Increasing the number of instances from 4 to 8 did not signifi- 

t 

cantly affect overall concept mastery for either fourth- or 
sixth-grade children. 

(2) Increasing the number of instances from 4 to 8 significantly 
improved recognition of concept non-examples for fourth-grade 
children. 

(3) Emphasis of relevant attribute values significantly increased 
overall concept mastery for fourth-grade children. The increase 
in overall concept mastery for sixth-grade children was not 
significant. 

(4) Emphasis of relevant attribute values significantly increased 
recognition and production of the names for attribute values by 
fourth-grade children. It also significantly increased recog- 
nition of the names for attribute values by sixth-grade children. 
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Instructions to Students 

My name is . I work at the University of 

Wisconsin in Madison. At the university, we are trying to find out 
the best way to help children learn geometry. We have written four 
lessons about geometry. Today you will study the first lesson, and 
Tuesday, Wednesday, and Thursday you will study the other lessons. 

On Friday, Monday, and Tuesday we will give you tests with questions 
about the things you studied in the lessons to see how well you 
learned them. By doing the best job you can on these lessons and 
tests, you will help us find out how to teach geometry so it is 
easier to understand. 

I am going to hand out the first lesson now. When you get your 
booklet, please do not open it until I tell you what to do. When I 
call your name, you may come up and get your booklet and also take 
one of these pieces of green cardboard. 

Now turn to the first page of the lesson where it says "word 
list." (Read the directions on the page to the children. Then pro- 
nounce each word and have them repeat it aloud with you. Briefly 
explain meaning of represent , e.g. a picture of a cat represents a 
real cat. Then, go through list in random order and ask them to 
find the word you pronounce. Provide feedback by telling number of 
word after pause.) 

(Hold up a copy of Lesson I turned to the first page of the 
lesson, so the children can see it while you give the following 
instructions . ) 



101 



This lesson may be a little different than other lessons you 
have studied. The pages in your lesson will look like this. This 
side of the page (point) has questions for you to answer. The other 
side of the page (point) gives the correct answer for each question. 

When you do the lesson, you should cover the answers with the 
green cardboard, like this (demonstrate). After you write your an- 
swer to the question, move the cardboard down iust far enough to see 
if the answer you wrote was correct. Then write your answer to the 
next question, check your answer, and so on. When you finish with 
one page, turn to the next page, cover the answers and then go on 
in the same way. 

Now turn to the next page in your booklet and we'll do the first 
page together. Remember to cover the answers. 

(Read frame 1, allowing time for children to write in their 
answers. Prompt them when to move answer cover, and read correct 
answer aloud to them.) 

If you make a mistake on a question, cross out the wrong answer 
with a single line like this (demonstrate on blackboard) and write 
in the correct answej; below it (demonstrate on blackboard). By cor- 
recting your answers this way, we will be able to find out what 
questions were too hard or confusing, and we can write better lessons 
next time. Suppose on the last question, "What are the names of the 
points?", I had answered "P, Q. and R." (Write P, Q, and R on board.) 
How should I correct it? (Have children tell you how to correct it 
and follow their directions.) 




Does anyone have any questions? You will go through the rest 
of the lesson by yourself now. If you have any questions or come to 
any words that you do not know, raise your hand and I will help you. 
Remember to correct your answers by drawing a single line through the 
wrong answer and writing the correct answer below it. 

Write the time it is now (tell them the correct time) on the 
front cover of your booklet where it says "starting time. lhen, 
finish the lesson. When you are done, write the time that you finish 
on the front cover where it says "finishing time." Take your time 
in doing the lesson so that you will understand it and be able to 
answer questions about it later on. 
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APPENDIX B 

TEACHER QUESTIONNAIRE: 
KNOWLEDGE OF GEOMETRY CONCEPTS 





i 
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Knowledge of Geometry Concepts 

1. Prior to the experiment on April 28, 1969, which of the follow- 
ing geometry concepts had your class studied during the current 
school year? 



YES NO If yes, please give 
approximate date. 

closed curve 

simple curve 

plane figure 

parallel 

equal length 

right angle 

opposite 

adjacent 

number of sides 

quadrilateral 

parallelogram 

kite (in technical sense) 

trapezoid 

rectangle 

rhombus 




square 
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2. Prior to the experiment on April 28, 1969, what degree of mastery 
do you think the majority of students in your class possessed 
regarding the following concepts? 

Completely Some knowledge. Mastery 

Unfamilar but not mastery 

closed curve 

simple curve 

plane figure 

parallel 

equal length 

right angle 

opposite 

adjacent 

number of sides 

quadrilateral 

parallelogram 

kite (in technical sense) 

trapezoid 

rectangle. 

rhombus 



square 
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